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ABSTRACT 

To determine' whether admission interviews could 
differentiate commitment, 456 applicants from 2 medical schools were 
tested on the commitment, 456 applicants from two "medical schools 

'were'.tested on the Defining Issue's* Test, which measures, the amoynt of 
principled or postconvent ional moral reasoning. The participating 
institutions were two Israeli institutions : the Sackler School of 
Medicine , ,Tel Aviv University, and the Ben-Qurion University. In the 
Defin-ing Issues Test , '/tl^e sub jectS4are required to respond to a 
aumber of moral dilemmas , ' eac^followed by 12 moral issues relating 
to it*. The issjues are criteria for solving the dilemma and are; 
designed to represent The various stages of' moral development, 
according to Kohlberg's theory. Each subject rates all ttie issues in 
terms of theif importance in solving the dilemma, and rank orders the 
four most important ones. No diffe^Bnce was fojind between the "scores 
of the accepted and the rejected applicants of, the school usitog' 
traditional scholastic admission criteria. On the* other hand, a great 
difference was shown in the school that admits, students, for their 

.personal characteristics as assessed by interviews. How.ever, only 
moderate 4 ' correlation was fountf between the test and -interview scores. 
It is suggested that moral reasoning is & key ^oncept iti medical 
professional behavior and is correlated wi,th clinical performance. A 
possible use-of the Defining Issues Test in. the 'student selection 
process is to increase the pcedictive value of the- admission .process 
while decreasing its : /cost . Possible explanations for the difference 
in the response rate between admitted and* rejected applicants in both 
institutions are offered . (SW) 



****************************************************************** 

*' ^Reproductions supplied by EDRS are the best that^cafi be made . * 

* , *from the • original document . * * 

********************************** **>******,***************************** 



ERLC 



CO 

« — i 
o 



5 * 



MORAL REASONING AS A CRITERION'- 
FOR ADMISSION TO MEDICAL S£HOOL 

' Dan E. Benor , M.D., Neta Notzer, 
T. Joseph Sheehan, Ph.D. and Geoffrey R. Norman, Ph.D. 



Subtitle: MORAL RFASONJNG and ADMISSION 

* 

Correspondence: Dan E. Benor, M.D. , Faculty of Health Sciences, 
Ben -Gurion University, Beer-Sheva, Israel 



V 



Part of this paper will be presented in the AERA Annual Conference, 
New York, March ]982. 



US 0EPAR7MENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 

EDUCATIONAL PfSOURCFS >NfOR\1 A T (ON 

«/r his fb>' i 



' * n Mis t»fn t*prvi'A «1 



','lfjlfl |l f..| ,, 

ri'fjfO-Jn. f i r ,i f „. 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HA.S BEEN GRANTED BY 1 

— /yY*< 



'3* 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) ' 



■'4 



9 

ERIC 



/" 

/ 

/ 



I 
/ 



Jk.- 



Authors' descript i'Dn 

\ ■ ; J - 

/ ' < % ! . / ^ j 

D/t . Benor rs the Head, Planting and Evaluation Unit, Faculty of 
Heal/h Sciences, Ben-Gurion University, Beer Sheva, Israel, and a 

. / • ' . • 

Senior Lecturer, in \ am*i ly Practice in the same institution. Mrs. 

/ . ^ s- 

Nozer the Director, Department of Medical Education, Sackler School 

of Medicine, '/el Aviv University Israel . Dr. Sheenan is Professor 

- i 

and Head,, Department of Research in Health Education, University of 
Connecticut Health Centre Farmington, Coripecticut . Dr. Norman is 
Assistant Professpr, Program for Educational Development, Faculty of^ 
Health Sciences, McMaster > University , Hamilton^ Cana^fe. 



\ 



The' search for good predictors of professifjnaL performance which can ' 

he used for student selection is ceaselessly carried on. In medical 

education the % trndit ioiul admission criteria, which are by and large- cognitive, 

have been repeatedly challenged. A multitude of studies' pointed out that . 

pjre-medical grade-point average (CAW) , old and new MCAT, entrjjnce examinations 

and other*, are poor predictors of clinical performance (Wingard § Williamson, 

1973; Murden at al., 1977), Further, there is increasing evidence that non- 

i 

cognitive pa^amotcrs such, as interpersonal skills, social awareness and * 
moral behavior .are indeed better discriminators .between good and poor 
ph^icians (Rezle'r, 1973; Margolis f, Cook, t 1974; Wagoner 5 Gray, 1979). 
Y*t such non-cognitive attributes are difficult- to measure quantitatively, 
and believed by some to be unassessable by a pencil-andrpaper instrument . 

(Werner et al . , 1977). Hence, those institutions which attempt .to assess 

* s 

non-cognitive qualities of applicants have to rely on ^ersonail interviewing 
in spite of its questionable reliability and validity (Gordon 5 Lincon, 1976;.' 
Milstein cfal., 1980;, Greer 5 Aronson, 1980), jand its cost in time and/ ° 
manpower.* n * ' & 

Challenged by the need for a valid* reliable and simple instrument to y 

support the student selection process; the authors chose the Defining Issues 

/ ■ • * 

Test^DlJ), developed by'' Rest (1979) on the basis of Koh 1 berg 1 s moral ; \- ' 

development theory (Konlberg, 1976). There were several reasons for the . . , 

choice of this particular t£:-t„ First, moral behaviour is 3 key donc5pt in 

any interpersonal relationships, which in turn are«cfentral to medicine c It 

is a closely attached concept to social sensitivity and -empathy (Mead, 19^4; 

Baier,^ 1973) and related to social adequacy and interpersonal effectiveness « 

.(Grief ft flogan, 1973). Others have addfcd to the definition of morality "the 
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care for thd individual as well as for the Sntire soc'iety" (Hogan, 1973), 

All this prr.i;, f descriptions closely corresponds to tTie "principled" or 

* J. 1 i * 

pnst-^>nvonticin;il stage of moral development as suggested by Kohlberg (1976). 

This is the level in\hich" moral issues are self-accepted and internalised. 

\ - ■ ■ 

rt has been further, subdivided \into the Stage -of "morality o£ contract and 

i 

ot democratically accepted law" (5A) ; of "morality of individual principles 
• . . \ 

*of conscience*" (5b>; and of universally valid ethical principles of "justice « 

\ ■ •• : 

and perfectionistn" (6) (Kohlberg, 1976). The behaviour in these levels, which 

. ' /- I* ' * \ . ' " . 

are the ones measured by the DIT, is internally motivated, in contrast with 

\ v * * . 

the d^ternal motivation of compliance with social norjns ("moratoty off 
conventional role-conformity") 6f the earlier stages. The importance of- 
internalised mora^ty^for medicine is T^eyon^ question. True, the DIT measures 
^judgement" or, developmental stage rather than actual behaviour, yet<such 
development njay be regarded as a prerequisite for the actual behaviour, and , 
thus for care and empathy. m . j 

A second reason for focussing on the DIT' is its well-established 
reliability , and validity, shown in many studies (b e oth intra- and tr*ans- ; 
dultural) (Rest, 1979; Rest et al. y 1977). A third reason* in favour of' i "° 
this instrument is it§ simplicity; it is a paper-and-pencil test, mechanically 
scored.. Those two 'last arguments single out the DIT from some' other ^ 
innovative empathy measuring instruments (Werner et "at. 3 1977)., " . . 

The research hypothesis was that the applicapts admitted to aft 
institution which selcLts its students for their personal virtues by- personal 
interviews will also score higher than the rejected ones qn the DIT. No- 
such difference was expected between -admitted and rejected applicants to, 
an institution which uses the traditional pre-medical . scholastic clchievenfcntrs 



and intxi'I ijjonV e as ihe nrnn admission criteria. It was further hypothesised 
that r'nc interview scores will correlate with the HIT scores to a significant 
degrde . ^ , ** 

. v \ 

BACKGROUND 

f • * 

Two Israeli institutions participated in the study. One is the Sackler s 

School erf 1 Medicine, Tel" Aviv University" (STA) , which selects its students from 

a pool of about 1 200 ap^Ucants per year on the basis of previous schbtfastic 

achievements and performance on a psychometric test. Only applicants with 

a national matriculation examination average of 80 percent, or better are 

considered. For those a combined 'performance score 1 is computed from, the 

- matriculation score (40 percefit) and the psychometric scores (60 percent). 
The fop 80 performprs are* admitted. ^The selection process* is automatised, 
economical and rapid. STA exemplifies an admission policy whiclj values 
intelligence, cognitive abilities and achievement orientation. Basically, 
it is* in accord with the policy of the majority of Western medical schools. 

The faculty of Health Sciences > Ben-Gurion University (BGU) has a 
rathe? complicated admission prpcess which Jias been 1 described in detail 
elsewhere (Antonovsky, 1976). It emphasises' personal characteristics, 

\ interpersonal skills and orientation toward the community as expressed by 
previous behaviour, and* doun-plays scholastic achievements. 4 Onlfl B gf ade^ • * 

* ' ' . - ■ J 

> (80 percent) on any two -of the six national matriculation exam*nation.s are 
*♦ * - , * • 

required an8 the overal average is ignored. The* top 30*0 or so scorers on >• 
a psychometric test out .of 1200' applicants*per year are interviewed. at length. 
The' top scored interviewees (about 90) are re-interviewed by another 'team. t 
The personp-f characteristics assessed are integrity; empathy; intellectual 



curiosity and flexibility - , intelligence; insight; and ca're fjr, involvement 

4 

with and drientation toward th<T"community. Each of the two 45 to 60 s minute 

. ; • ! 

interviews includes both structured and unstructured moral dilemma situations 
within a clinical context. °The final ^election of 50 students per ye^r is 
based entirely upon the interview Scores, BGIT thus, exemplifies those schools 
which value non^cognitl ve attributes and has gone further than .many 
institutions in challenging the predictive value of the previous scholastic 
achievements. • 

METHOD • . " * * 

. The DIT was offered to*a*l'l the BGU- applicants who were qualified for 
intervievung, *i.e.,.the top scorers on the psychometric test (N=319) , Qut 
of these 240 agreed to participate, and took the test immediately after the 
first 'interview. (Response rate of 75.2 percent. 1 The interviewers were 
not aware of the study, and are not familiar with the DIT. The DIT was also 
offered to all the 1166 STA applicants who wished to cooperate, and was taken 
immediatei> after their psychometric test. Only those who were eventually 

' " i r . ' , r' 

included among the top 1 316 scorers* on th& psychometric test wdre taken into 

consideration. .Out of .these 216 responded (response rate k of 67.4-per£ent) . 

' ' ' 

Each of #he two groups, BXIU and STA applicants was further subdivided into* 

' " / . • ** * * 

•i 

admitted and rejected. It muy be noted that .all the 50. students admitted to 

BGU were included in the study^w^ile o^ly 57 out of the 80 admitted to 1>TA^ 

were considered; the remainder had a lower psychometric score which was 

f 

compensated. by a high matriculation 'examination average. 

9 * f 

/ 

All the sublets were administered Rest's DIT. In this test *the subjects 

<3 

were required to respond to a number of moral dilemmas, each followed "by 12 



' , • ■ ^ .' \ 

moral issues relating to it. Ihese issues are, in matter of. fact, criteria 
fou-solving the dilemma. These wen* designed to represent the various- stages 
t %€ moral development according to Kohlberg's theory. Each subject had to * 
ra^e all the issues in terms of their importance in solving the dilemma 4 , 

and then to rank order the four .most important ones. On the basis of this 

* * • ' * 

selection and ranking a moral judgement score (P-score) was computed, 

reflecting the amount of principled or post -conventional^ reasonin^^Qf the 

examinee (Kohl berg's Stages 5 § 6) . 

". / » * 

. • In the present study both the dilemmas and the issues were translate^ 

1 - * - • r 

into Hebrew by the 1 r.esearch team. The translation" was checked by re- "V 

translation into English byan independent assistant: A number^of minor 

changes were introduced, all of which were approved by the author of the 

test. Tor examples, the mimes, nationalities afld occupations of some of the 

figures in the Stories were changed^ in order to maintain situations which 

woyjd be as familiar to the Israeli subjects as they are to the North American 

ones 'in the .English version. , For the same reasons the |horter form of the 

i * 

DIT was used,, offering three dilemmas instead of six. The short form was 
proven td 'yield almost the same results as the full one, and is also" , 
recommended by the author of th^test (Res?t, 1979).- 

Each protocol was' Checked for internal inconsistencies and the' 
incons-istentf^subjects were excluded. Further, ^subjects who tend to en'dfJrke 
meaningless yot lofty sounding items were also discarded ^the procedures are 

• detailed M in the DI t T manual; Rest , *1979) % This exclusion resulted^ in%a 
further reduction of the study population by 17.1 percent (ranging from 16.6 
to 19.1 percent in the various subgroups). Thus, the final study population 

"was 199 l BGlh applicants of whom 38 were eventually admitted, and 179 STA 



candidates of whom* 44 were ulputted. It may be noted that the response/^ 
rate in the two -admitted subgroup's was considerably higher than in the two 



rejected ones - . 


Table 1. summarises %)l 


e' study population* 




' Table 1 about 
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here ' • " 


« 

RESULTS 


\ 9 


x ■ 



I 

The P .-score over the entire stu'ty^population "was 41.00 ± 13.8 XTab.le 2-), 

which corresponds to- £ho scores: of^college .students elsewhere '{ReSt bt at., 

% 

1977,; Rest 1^)79)* The^ subgroup admitted to BGU scored significantly higher 
(50. G9 4 + 17.0) than the other subgroups: rejected from BGU (p <\001); 
admitted to STA-(p < ".0p5)'; and rejected from STA (p < .001) (Table 2)." 

4 * 

The differences .were higher injstage'5 of the moral development than in stage 

* ■ " , ** • 

6 (Table 2) . The analysis of variance indicates that the nominal* variable 

of admittecf to/rejected* from BGU accounts for most of the explained variance. 

There was-no effect to the intelligence, to the choice of school to apply' 

: f # 

(BGU. STA or both), and' to admission to/re jection v from STA. 



Table 2 about- here 



As the presented results might hav$ been biased by the "different response 
rates in the various subgroups, a conservative *approach was taken. TH^ee new 
hypothetical subgroups were creited for admitted to STA, r^jeefced f^om STA 
and rejected from £GU, assuming that : a) all., the members of the subgroups 
■respond;' b) all the 'new 1 ' subjects, who actually, did hot respond, scored as 
high a$ those admitted to BGU and had the same distribution; and c) none of . 
the 'new' subjects was* discarded ' because, of .inconsistencies. The results 



-r. 



indicate that ihc superiority of'the subjects admitted to BGU in regard te 
the P~score is a true one (Tcfble. 3). ' # , 



Table '3 about here 



As the interview score* aro an ordinal variable rather than an interval 



1 " 4 

one, the correlation botwoon P-scorcs and tho interview scares was studied 

* by computing^ non-parametric Correlation coefficient ( Randall 1 s tau) . Tho 

" * J 

results indicatd quite a moderate, although significant correlation (r = 0.19, 
p < .OS) 4 . Similarly moderate relationships Iretween these variables emerged ■ 
from Contrasting the higher and the lower thirds on' the P-sdore variable vs. 
high and low scorers in the interviews. Low scores* w^re defined as 6 to. 10, 
which practical ly mean "not to admit" ;^high score^ were 1 tb 3 on the same^ 

• scale, which indicate a strong recommendation for acceptance. Both Kendall's 
/-taus b and c were '.11 (p <..05). (Fa/giire l.j * ' 



Figure 1 about here* 
■ ^ ' _L_J . ^ » , . 

DISCUSSION ' . 

' T ' . . 

In trying to select students who possess the personal characteristics 

P believe^ to be required of si good physician, the BGU admission process has 

indeed sorted oat the higher morally developed applicants. These were not 

identified in STA, Using psychometric and scholastic criteria. . ^ 

f . - . • 

, Surprisingly, t\m pronounced difference between the admitted. and the' 

rejected BGU applicants was not accompanied by the expected high correlation 

between the interview and the moral reasoning scores. SeveraJ speculative 

s * 

explanations may account for this discrepancy. It might bave been that the 



r 



moral reasoning served u-sjaj^h^ excluding lower developed 

' t : ? t * , , \ 

candidates. Acceptable applicants were then ranked on different, institutional 

specific criteria. Alternatively/ the correlation achieved might be the out-- . 
- - - - "" - m * V ' 9 , 

qorae\6f an intervening variable which correlates with both-moral development 

, - : ' - { • ' ' ' : 

and interview # scores. Such a variably - may be the socialibility parameters 

• . \ • ' 

(Rezler, 1973). Another possibility is\that the reduction of th<£ correlation 

steps from the lack- of sufficient in^oVr^r rel iability among the interviewers, 

(Gordon Li neon. 1976; Milstein^t al\ 3 198&;. Greer S.AroYison, ,1 ; 980); in other 

' words, the interviewers were able to agree /on who is sufficiently morally 

* developed, but not on his or her numerical grading. The available data 

cannot differentiate between these possible explanations. ■ > * 

y A more -important question is. to wjiat extent moral" reasoning actually 

predicts clinical performance.- The relation between morality and empathy, * 

care, interpersonal relations and 'social, consciousness 1 ; aJLl so central to 

medical -practice, has already bfeen mentioned above (Mead, J.934; Baier, .1965} 

Grief f, Hogan, 1973; Hogan, 1973; Mar^clis § Cook, 1974; Wagoner £ Gray, 1977), 

Recent? data suggest that, moral reasoning clearly correlates with clinical 

performance across many approaches to the data-;* and that high "moral ^reasoning . 

"virtually excludes the possibility of poor performance" (Sheehan et al tJ # , 

1980). Moreover, the state, of development of morat reasoning irk adults *does 

not tend' to change along N time ( Kohlberg , .1976) , and actually did W change 

' • . • \ 

■> * \ 

during three years of medical school (Sheehan -et al. , '1981^ • - 

The fconspici^fds difference in the response-' rate between the 'admitted 

and the rejected applicants in both' the institutions is a rather strange 

phenomenon. It should be remembered that at the time* the test was faken* 

\ ; \ 

neither the applicants nor the admission cpmmit'tees could have knowa V who will. 4 



eventually he admitted, which, m^nns a real 'double blincft situation, "it* V v 
may be assumed that- th'e^response, ;rate reflects an attitude towards, research, 
and thus hints upon a personal ity % trait which had npt'been explored either 
by, this study or by the admission interview. This phenomenon i$ worth more 
'careful at teat Ion. ^ 



Insofar as morality is concer.ij.cd, tW data presented here, do not suggest 

a self-selection of highly moraMy developed individuals for the medical / 

profe^ion. . Both the entire stydy population and *eeeh bf the subgroilps 

scored abopfthe Same as .col lege* students elsewhere in any professional 

9 • ' <. 

traek >(Rest,' 1979)! It might thus be erroneous to select medicat j>£udejTts * 

Cxpon their cognitive abilities alone, while assuming sufficient moral 

development indicated by their professional choice. • * ' \ ' : 

The data which have been* presented offer for consideration a simple, ^ 

• t 

reliable and probably criterion valid instrument in selecting' medical 
$ « t • ' * 

students. It may- supplement the existing cognitive instruments or it may 
serve as either a back-up or a scjpening procedur e in asses^m nn- t^ of » inte>-n - ; — 
personal skills, "enabling /the interviews to b c - ' i nfl ii directed to .the 
institrtltionn I specific objectives. The instrument has, the potential of . 
increasing the predictive value ,of the'*admission process while deceasing 
its3 cost. A replication of this study and a longitudinal follow-up of the 

: • • ' • • • T ■ ' • 

admitted applicants may throw additional li,ght .on the validity of t-his 
recommended procedure. ' ^ • « • * , 

SUMMA RY -** . ' " 

To determine whether admission interviews could differentiate applicants' 

' V J ' . ' ' ' , ' •* ' * ' 

on- their personal qualities such as integrity, empathy and commitment , 456 



-10- 

applicants from tjwtf* mgdical schools were tested on the Defining Issues Test 
(DIT) which prf>asures the amount of principled,' or post-conventional moral 
reasoning*: ^o jdifference was found between the DIT scores of the' accepted 
and fcHe rejected applicants of the school where the admission criteria, are 
the traditional scholastic on£s. On the other hand, a great difference 

* > • ■ . N c * 

was shown in the school which Admits students, for' their personal characteristics 
as assessed by -interviews . Yet onlv moderate 'correlation was shown between 
the DIT and the interview scores. Since moral reasoning i^/a key copcept in 
medical professional behavior and is correlated with clinical performance, 
the findings deserve special attention. A possible use of the DIT in the 
student selection process is 'discussed. * , | 
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*■ TABLE 1 v . 

* c 

The* Study Population 





' Actual 

N 


Respondents 

N % 


Excluded 

N % 


Study Population 
N % erf # 
* original * 


BGU applicants 


319 


240 


. 75.2 


'41 


17.1 




62.4 


'admitted 


,50 




94.0 


9 


} 19.1 


38 


76.0 


rejected 


269 




71.7 


32 


16.6 


161. 


59.9 


7 STA applicants 


316 - 


216 


< 

•68.4 


• 37 . 


17.1 


179 


56.6 


' ' admitted 
; ' rejected i 


57 

259 ' . 


53.- 
163 


93.0 
62.9 *' 


" , 9 

s 


17.0 

■17.2 y 


44 • 


77.2 
52.1 


< -* Total 


„ 635 

* 


456 • 


71.8 - 


78 


' 17,1 


378 " 


59.5 



Excluded because of -internal inconsistencies. 
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l& ( ' TABLE 2 



Principled moral 


reasoning score (P-score) in thevarious 


groups: 


Means, standard 


deviations, and between 


group© t val 

/ 


ueff 

> 


Group 


, N 


P-score 
(Stages 5 & 6) 


Stage 6 


Stage 5 


Entire population 


378 


41.00±13.8 

• 

4 




*• 


BGU^applicants 


199 


41.41+14.3 






admitted 


38 


50.08±17.0 


14.73±8.3 
$ 


35.35±13.9 


rejected * 


161 


39.36±12.8- 


11.62+6.8 


27.74+10.9 

f 


STA* applicants j * 


i79 


* % 

f - • 39.88±13.0 






admitted 


44 


39.47±12.4 


■ 13.18±9.0 


: ■ > 26. 29*10 f 6 


• > rejected * 


' 135 


• /' A0.02±13.2 


■ 12.67±7.4 


27 / /35+10.8 


Between groups (t values) 




- 




Admitted BGU/^A* 
Admitted/rejected* BGU 




3.25**" " 
4.35*** - * 


.81 "S 
2.43* ' 


" 3 . 35*** 

f 

3.67*** 


Admitted/rejected STA 

, — ■ ** L I 






.14 

r 


- .80 

i 


* p< .05- . 










** p< .01 










*** p < .001 






* 
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TABLE 3 • ' ' &> ' ^ 
Principled moral measoning score (P-score) in the hypothetical 
groups*: Means, Standard deviation and between groups t values 



Group 


real N 


assumed HI? 




real' 


assumed . '* 










P-score 


P-score 


BGU applicant^ • 




6 




• 




admitted 


38 






50.08±17.0 




' rejected 


161 


237 


* 


0 

39.36±12.8 


- ' .42/80±13.8 














STA applicants 










1 

i 


admitted 


44 


48 




• 39.47±12.4 
i 


. 40.35±13.5 


* • 

rejected 


135 . 


231 v 




40.02+13.2 

. il 


44.17±14.5 


Between Groups (t values) 










Admitted BGU/STA % 








' 3.25" + - 


• " 4.82 +t -' , 


• 

Admitted/rejected BGU 






+++ 

4.35 


2-9.1' 


Admitted/rejected STA 




0 


- .57 


-1-.67" 



The hypothetical groups are comprised Of all the actual respondents plus 
the non-responding members of the. group . For the* non-responding pjnes a 
P-score equal to the mean and standard deviations of the admitted to BGU 
assumed. 



— fr- 
ail 

was 



t p < .05 

;tt ..oi 

tt+p < .001 



S 



* * 



